# Home Care Monitoring Box Copyright © Quectel Wireless Solutions Co., Ltd. 2026. All rights reserved. --- This project is a home care monitoring sample built on the Quectel Pi M2 smart mainboard. The system captures live video through a USB camera and runs the YOLOv8n-Pose human pose model on the on-board NPU for multi-person skeleton detection and fall determination, combined with sound energy monitoring through the on-board microphone, audible and visual alarms, and event recording. It demonstrates the end-to-end pipeline of "video capture → NPU inference → fall determination → audible/visual alarm & event recording". When a fall is detected, the on-board red LED lights up and the speaker plays a siren, and an audio-annotated video clip covering the moments before and after the event is saved automatically; the GUI shows the skeleton overlay, frame rate, inference latency, person count and sound level in real time. > It is recommended to insert an application photo here (UI running view + photo of the whole device) as the cover. ## Developer Resources ### Accessory List > Note: tips — The accessory list is for reference only; actual requirements may vary with the development environment. [电子表格] ### Accessory Photos > It is recommended to insert a photo of the accessories here. ## Quick Start ### Preparation The Quectel Pi M2 smart mainboard ships with the Debian 13 image by default, so there is no need to flash the image again — simply follow the steps below. The NPU runtime library, the Python 3.12 environment and all dependencies are installed automatically by the one-click deployment script bundled in the project. ### Hardware Connection Complete all of the following hardware connections: 1. **Camera**: plug the USB camera into the USB-A port of the mainboard. Use v4l2-ctl --list-devices to check the device index (on the reference unit, PC Camera = /dev/video73), and fill it in the configuration file later. 2. **Sound capture**: the microphone built into the camera is used for sound capture by default (plughw:1,0 in arecord -l), no extra wiring is needed; the on-board es8389 microphone serves as the backup. 3. **Speaker**: connect the speaker to the audio output jack of the on-board es8389 sound card (corresponding to plughw:0,0); the power amplifier enable is completed automatically by the kernel, with no software intervention required. 4. **Display (optional)**: connect an HDMI monitor to view the GUI locally; without a monitor, run in headless mode and check event logs and recordings remotely over SSH. 5. **Network and power**: connect the Ethernet cable first, and finally connect the Type-C power adapter and power on. > Note: tips — After connection, verify that the camera is present with v4l2-ctl --list-devices, and verify the recording/playback devices with arecord -l and aplay -l. If the camera index differs from the configuration, just change camera.index in config.yaml. ## Project Implementation ### Get the Code Open a terminal on the mainboard and clone the project code with git. ```plaintext sudo apt update sudo apt install -y git git clone https://github.com/Quectel-Pi/demo_fall_detect.git cd demo_fall_detect ``` After the command completes, enter the demo_fall_detect directory, which contains all source code of this project. ### Environment Setup Run the one-click deployment script bundled in the project to automatically install a standalone Python 3.12 environment, inference dependencies and the RKNN runtime library: ```plaintext sudo bash deploy/board_setup.sh ``` The script supports both **online and offline modes**: in online mode, Python 3.12 and pip dependencies are installed via uv; in offline mode, the wheels and offline Python packages shipped with the bundle are used. It also installs the NPU runtime librknnrt.so into the system directory and deploys the code to /opt/demo_fall_detect. ### Self-Test Before the first run, it is recommended to run the built-in self-checks in order to confirm that the hardware and software links work properly: ```plaintext /opt/py312/bin/python3.12 selftest.py # Offline self-test: post-processing / fall decision / siren generation, no camera or NPU needed sudo bash run.sh --test-alarm # Audible/visual alarm test: red LED + siren for 5 seconds sudo bash run.sh --test-audio # Sound calibration: prints ambient noise and sound level (dB) in real time ``` --test-audio can be used to calibrate the sound trigger threshold (the audio section in config.yaml) against the actual environment, avoiding false or missed triggers. > Note: tips — All on-board dependencies are installed into the standalone /opt/py312 environment created by the deployment script. Do not run this project with the system python3 (it raises ModuleNotFoundError). run.sh locates the environment automatically — always launch the program through run.sh. ### Run the Program ```plaintext sudo bash run.sh ``` run.sh automatically locates the deployment directory and the Python environment, sets up the desktop display permissions and starts the program. If the program starts properly, a "Fall Detection Monitoring System" window pops up, with the live skeleton-annotated picture on the left and the status metrics and control panel on the right. For scenarios without a monitor, run in headless mode: ```plaintext sudo bash run.sh --no-ui ``` ### UI and Features After the program runs normally, the status badge at the top of the window is a green "●Monitoring"; when a fall is detected it turns into a red "⚠Fall detected!", and the audible/visual alarm and event recording are triggered at the same time. [电子表格] ### How It Works System data flow: 720p@30fps camera capture → letterbox scaling to 640×640 → YOLOv8n-Pose on the RK3576 NPU (fp16, about 60 ms per frame) → post-processing outputs 17 keypoints per person → fall determination → triggers alarm/recording. **Fall determination** uses a scoring system, judging each detected person in the frame independently: - Prerequisite: the proportion of reliable skeleton keypoints of the person is ≥ 70%; - Feature scoring: features such as torso tilt angle, horizontal legs, landscape body aspect ratio, keypoints close to the floor and horizontal distribution are weighted and accumulated; a total score ≥ 3 is judged as a "fallen posture"; - Hard veto: when the torso is upright (tilt angle below 0.6 times the threshold, about 33°), it is rejected directly, filtering out false positives from upright-torso scenarios such as sitting, squatting and bending over to pick things up; - Auxiliary rule: a fast hip drop within the observation window combined with an excessive tilt angle triggers the determination directly; - Debouncing: the alarm is triggered only after the fallen posture persists for 15 consecutive frames (about 0.9 seconds), and is cleared only after 30 consecutive frames of normal posture. **After an event is triggered**: the on-board red LED lights up and the speaker plays the siren in a loop (10 seconds by default; the alarm continues while the fall is not recovered), and an audio-annotated recording covering 4 seconds before the event (10 seconds in total) is saved automatically to the records/date/ directory, with the index written into records/events.jsonl. Abnormal sound events (sudden volume increase) only trigger recording, without the siren. The determination thresholds, camera parameters, sound thresholds and recording duration can all be adjusted in config.yaml; tools/fall_feat_debug.py can be used to print the torso tilt angle and other feature values in real time, assisting on-site calibration. ### Application Demo > It is recommended to insert an application demo video here (fall-triggered alarm + event recording playback). ## Frequently Asked Questions and Solutions ### Program cannot find the camera Symptom: ```plaintext Camera open failed ``` Solutions: - Confirm the camera device index with v4l2-ctl --list-devices; - Change camera.index in config.yaml to the actual device index (73 on the reference unit of this project). ### Person present but not detected / sitting false positives Symptom: a person is in the frame but the person count is 0, or a normal sitting posture is judged as a fall. Solutions: - Properly lower model.conf_threshold / model.kpt_threshold (defaults 0.35 / 0.40); - When the distance is too far so that keypoints are invisible, the determination is blocked by the "visible keypoints ≥ 70%" gate, which is a normal protection logic; - For sitting false positives, run tools/fall_feat_debug.py to view torso_angle and each score in real time, and fine-tune fall.torso_angle_threshold if necessary (default 55°, the upright hard veto is 0.6 times, i.e. 33°). ### Sound falsely triggered or never triggered Symptom: recordings are made frequently in a quiet environment, or clapping and shouting get no response. Solutions: - Run sudo bash run.sh --test-audio to observe the ambient noise floor and the real-time dB reading when clapping; - Adjust audio.abs_threshold_db in config.yaml according to the actual environment (default -10 dBFS, linked with a relative threshold 15 dB above the noise floor); - The default sound capture device is the microphone built into the camera (plughw:1,0); if it is occupied by the desktop audio service, the program automatically falls back to the on-board microphone (plughw:0,0). ### GUI fails to start Symptom: after running in a desktop environment no window pops up, and the program falls back to command-line mode. Solutions: - Start with sudo bash run.sh; the script already handles the display permissions under the Wayland desktop (DISPLAY / XAUTHORITY); - If the log says the Qt xcb plugin failed to load, install the missing system library: sudo apt install -y libxcb-cursor0 (the deployment script has a built-in automatic fallback installation); - For scenarios without a monitor, use sudo bash run.sh --no-ui directly. ### NPU inference error or model loading failure Symptom: ```plaintext RKNN run error / librknnrt version mismatch ``` Solutions: - Make sure librknnrt.so matches the on-board rknn-toolkit-lite2 version (re-running sudo bash deploy/board_setup.sh aligns them automatically); - Make sure the model files are .rknn files compiled for the RK3576 target platform (bundled in the models/ directory of the project); during debugging, a .onnx file can also be used temporarily for CPU inference (specified with the --model option). ## Technical Support and Contribution If you encounter any problems during use, feel free to submit a technical request on the Quectel official forum; our technical support team will answer you promptly. Project open-source address: https://github.com/Quectel-Pi/demo_fall_detect Issues and Pull Requests for code improvements are welcome!